Learning surrogate models for simulation-based optimization

We address a central problem in modeling, namely that of learning an algebraic model from data obtained from simulations or experiments. We propose a methodology that uses a small number of simulations or experiments to learn models that are as accurate and as simple as possible. The approach begins by building a low-complexity surrogate model. The model is built using a best subset technique that leverages an integer programming formulation to allow for the efficient consideration of a large number of possible functional components in the model. The model is then improved systematically through the use of derivative-free optimization solvers to adaptively sample new simulation or experimental points. We describe ALAMO, the computational implementation of the proposed methodology, along with examples and extensive computational comparisons between ALAMO and a variety of machine learning techniques, including Latin hypercube sampling, simple least squares regression, and the lasso.